Performance Analysis and Optimization of the Regional Ocean Model System on TeraGrid

The Regional Ocean Modeling System (ROMS) is a large-scale application code that can be configured for any region of the global ocean, ranging from local to basin scales, and is widely used in oceanography and atmospheric sciences. In this paper, we optimize the ROMS application for efficient execution on a grid environment, the TeraGrid. The strategy used to optimize the ROMS entails combining multiple communication steps and overlapping the communication with the computation as much as possible. In particular, the communication function, MP_Exchange is rewritten, and several new communication modules are added. To demonstrate the advantages of making this change to the communication function, we focus on optimizing the function step2d of the ROMS code, which dominates execution time of the code. Experiments are conducted on the TeraGrid resources at NCSA, UC/ANL, and SDSC. The experimental results show that the overall application performance is improved up to 36%.